- Definitions, motives, spectrum
- Current practices
- A selection of tools to improve reproducibility
- Challenges, standards & our role in the future of reproducible research
December, 2014
Empirical - Statistical - Computational
"An article about computational result is advertising, not scholarship. The actual scholarship is the full software environment, code and data, that produced the result."
- Claerbout and Karrenbach, Proceedings of the 62nd Annual International Meeting of the Society of Exploration Geophysics. 1992
"When we publish articles containing figures which were generated by computer, we also publish the complete software environment which generates the figures"
- Buckheit & Donoho, Wavelab and Reproducible Research, 1995.
Technical
Cultural & personal
Peng 2011, Science 334(6060) pp. 1226-1227
Click-trails compromise clarity
Scripted analyses support scientific integrity
The alternative to point-and-click analyses
"Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to humans what we want the computer to do."– Donald E. Knuth, Literate Programming, 1984
For example… Let's calculate the current time in R.
time <- format(Sys.time(), "%a %d %b %X %Y")
The text and R code are interwoven in the output:
The time is `r time`
The time is Mon 15 Dec 12:55:42 AM 2014
`
`
The machine-readable part
R: Free, open source, cross-platform, highly interactive, huge user community in academica and private sector
R packages: an ideal 'Compendium'?
"both a container for the different elements that make up the document and its computations (i.e. text, code, data, etc.), and as a means for distributing, managing and updating the collection… allow us to move from an era of advertisement to one where our scholarship itself is published" - Gentleman and Temple Lang 2004
`
`
Rmarkdown: lightweight document formatting syntax based on email text formatting. Easy to write, read and publish as-is.
The human-readable part
Payoffs - Eases collaboration - Can track changes in any file type (ideally plain text), and who made them - Can revert file to any point in its tracked history
Costs - Unfamiliar to most social scientists - Takes time to master
`
`
type: alert
Stodden (IASSIST 2010) sampled American academics registered at the Machine Learning conference NIPS (134 responses from 593 requests (23%). Red = communitarian norms, Blue = private incentives
Stodden (IASSIST 2010) sampled American academics registered at the Machine Learning conference NIPS (134 responses from 593 requests (23%). Red = communitarian norms, Blue = private incentives
An incentive to share data and code by acknowledging open practices with badges in publications. Currently used by Psychological Science
-Raymond, E. S., 2004, The art of UNIX programming: Addison-Wesley.
Presentation written in Markdown (R Presentation)
Compiled into HTML5 using RStudio
Source code hosting:
ORCID: http://orcid.org/0000-0001-7879-4531
Licensing: